Version 5 of Smarter Parser by Aaron Reed begins here.

"Understands a broader range of input than the standard parser, and can direct new players towards proper syntax."

[
CHANGES:
 -- Version 5: Removed deprecated features; added option to ignore all rules; adjusted responses to not have fantasy RPG elements & use more examples taken from the player's current environment.
 -- Version 4: Updated for build 6E59
 -- Version 3: added ? for help, more punctuation removal; corrected a typo; minor bug fixes.
]

Chapter - Definitions

Use MAX_EXPRESSION_NODES of 300.

The rejected command is an indexed text that varies. The original rejected command is an indexed text that varies. 

Suppress parser error is a truth state that varies. sp reparse flag is a truth state that varies.

Smarter Parser is a rulebook. 

Chapter - Empty Smarter Parser Rulebook

Use empty Smarter Parser rulebook translates as (- Constant NO_SMARTER_PARSER_RULES; -).

To decide whether sp_normal: if the empty Smarter Parser rulebook option is active, no; yes.

Section - Cleaning Up Punctuation

After reading a command (this is the Smarter Parser simplify punctuation rule):
	let tmpcmd be indexed text;
	now tmpcmd is the player's command;
	replace the regular expression "<;:>" in tmpcmd with ". "; [new players have sometimes tried to seperate commands using these punctuation marks.]
	if tmpcmd exactly matches the text "?", now tmpcmd is "help";
	replace the regular expression "<!?\\\/()\{\}['][quotation mark][bracket][close bracket]>" in tmpcmd with "";
	if the number of characters in tmpcmd is 0, now tmpcmd is "."; [To fix a bug where in certain cases an empty command causes the word/character counts to break]
	change the text of the player's command to tmpcmd.


Section - Try Harder to Understand


Before printing a parser error (this is the Smarter Parser trying harder to understand rule):
	now the original rejected command is the player's command;
	now the rejected command is the player's command; 
	follow the Smarter Parser rules;
	if the rule succeeded:
		if rejected command is not original rejected command:
			follow the announcing new command rule;
			now sp reparse flag is true;
	otherwise if the rule failed:
		now suppress parser error is true.

Rule for printing a parser error when suppress parser error is true or sp reparse flag is true (this is the Smarter Parser suppress parser messages on matches rule):
	now suppress parser error is false;
	do nothing instead.

This is the announcing new command rule:
	say "(retrying as if you'd typed '[rejected command].')".

Rule for reading a command when sp reparse flag is true (this is the Smarter Parser retrying input rule):
	now sp reparse flag is false;
	change the text of the player's command to "[rejected command]".



Section - Macros for defining rules

To decide whether stripping (regex - an indexed text) is fruitful, even within words:
	let original be the rejected command;
	if even within words:
		replace the regular expression "[regex]" in rejected command with "";
	otherwise:
		replace the regular expression "\b[regex]\b" in rejected command with "";
	if original is not rejected command:
		decide yes;
	now rejected command is original;
	decide no.

To decide whether input starts with (regex - an indexed text), even within words:
	if even within words:
		if rejected command exactly matches the regular expression "^[regex].*":
			decide yes;
	otherwise:
		if rejected command exactly matches the regular expression "^[regex]\b.*":
			decide yes;
	decide no.

To decide whether input ends with (regex - an indexed text), even within words:
	if even within words:
		if rejected command exactly matches the regular expression ".*[regex]$":
			decide yes;
	otherwise:
		if rejected command exactly matches the regular expression ".*\b[regex]$":
			decide yes;
	decide no.

To decide whether input contains (regex - an indexed text), even within words:
	if even within words:
		if rejected command matches the regular expression "[regex]":
			decide yes;
	otherwise: 
		if rejected command matches the regular expression "\b[regex]\b":
			decide yes;
	decide no.

Chapter - Printing Results

Section - Definitions

The pre smart parse text is an indexed text that varies. The post smart parse text is an indexed text that varies.

Section - The Print Rule

To print explanation for (why - a rule):
	if sp-debug is true, say "(smart parser debug: printing message for [why])[command clarification break]";
	if there is a therule of why in Table of Extended Smarter Parser Messages:
		choose row with a therule of why in Table of Extended Smarter Parser Messages;
		say "[pre smart parse text][message entry][post smart parse text][line break]";
	otherwise if there is a therule of why in Table of Smarter Parser Messages:
		choose row with a therule of why in Table of Smarter Parser Messages;
		say "[pre smart parse text][message entry][post smart parse text][line break]".

Section - Applicable Examples

To say get noun example:
	let noun_example be indexed text;
	if the number of visible things > 0:
		now noun_example is "[a random visible thing]";
	otherwise:
		now noun_example is "flower";
	say "[noun_example in upper case]".

To say get direction example:
	let randdir be a random sp_viable direction;
	let dir_example be indexed text;
	if randdir is a direction:
		now dir_example is "[randdir]";
		say "[dir_example in upper case], for example";
	otherwise:
		say "NORTH, for example".
		
Definition: a direction is sp_viable if the room it from location is a room.

Section - Tables

[This would seem an ideal situation to use the Table Replacement feature, but unfortunately it currently doesn't allow for replacing "rule" columns.]

Table of Smarter Parser Messages
therule		message 
a rule	a text
with 1 blank row.


Table of Extended Smarter Parser Messages
therule		message 
a rule	a text
with 1 blank row.


Chapter - Rules


Section - Standardize


A smarter parser rule when sp_normal (this is the standardize apostrophes rule):
	replace the regular expression "[apostrophe](m|re)" in rejected command with " am"; 
	replace the regular expression "[apostrophe]ll" in rejected command with " will"; 
	replace the regular expression "[apostrophe]ve" in rejected command with " have"; 
	replace the regular expression "[apostrophe]" in rejected command with "".


Section - Signs of Confusion


A smarter parser rule when sp_normal (this is the signs of confusion rule):
	if input contains "(confused|lost|huh|understand|dont know|going on|no idea|wtf|help|hint|instructions|directions)" or input contains "(how|what) (do|should|can)":
		print explanation for signs of confusion rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
signs of confusion rule		"Try typing LOOK for a description of your surroundings. You might try get more details about something you see by trying a command like EXAMINE [get noun example] to get more details. Any compass directions indicate exits which you can use by typing [get direction example]. Type INVENTORY to see a list of what you're carrying. You can also TAKE or DROP some things, OPEN or CLOSE others, and take other sensible actions."


Section - Signs of Frustration


A smarter parser rule when sp_normal (this is the signs of frustration rule):
	if input contains "(stupid|dumb|moron|idiot|lame|duh|retard|suck|screw)(ic|ed|s)?" or input contains "(hell|ass|bored|boring|dick|piss|die|fuck|shit)":
		print explanation for signs of frustration rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
signs of frustration rule		"I'm sorry if you're feeling frustrated. If you like, you can type SAVE to store your progress to a file, then RESTORE to come back to it later. In the meantime, you might try searching the web to see if there are hints available."


Section - Too Many Words


A smarter parser rule when sp_normal (this is the too many words rule):
	if the number of characters in the rejected command > 1 and the number of words in the rejected command > 6:
		print explanation for too many words rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
too many words rule		"You typed a rather long command and I didn't understand it. It's better to stick to simpler things like TAKE [get noun example]."



Section - Stripping Niceties


A smarter parser rule when sp_normal (this is the stripping niceties rule):
	if stripping "please" is fruitful:
		print explanation for stripping niceties rule;
		rule succeeds.

Table of Smarter Parser Messages (continued)
therule			message
stripping niceties rule	"Your politeness is appreciated, but you don't have to type words like please to play! "



Section - Standardize be verbs


A smarter parser rule when sp_normal (this is the standardize be verbs rule):
	replace the regular expression "\b(had |has |have )?(is|am|are|be|being|been|was|were)\b" in the rejected command with "_be".



Section - Who are you

["Who said that," "Who am I talking to," "Who are you," etc]

A smarter parser rule when sp_normal (this is the asking who are you rule):
	if input contains "who(s)? (_be )?(i )?(talk|speak|there|this|said|you|that)", even within words:
		print explanation for asking who are you rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
asking who are you rule	"I'm the parser-- I translate what you type into the story world. You have to use specific commands to talk to me though. Try typing HELP or ABOUT for more information."


Section - Who am I


A smarter parser rule when sp_normal (this is the asking who am i rule):
	if input contains "who _be i":
		print explanation for asking who am i rule;
		now the rejected command is "examine me";
		rule succeeds.

Table of Smarter Parser Messages (continued)
therule			message
asking who am i rule	"You are the main character of this story. Normally, you'll want to phrase your input as commands, not questions. I've taken the liberty of demonstrating below."


Section - Where am I


A smarter parser rule when sp_normal (this is the asking where am i rule):
	if input contains "where _be i" or input contains "(examine|x|look) (here|this place|place|room|area|around)":
		print explanation for asking where am i rule;
		now rejected command is "look";
		rule succeeds.

Table of Smarter Parser Messages (continued)
therule			message
asking where am i rule	"To get a description of your surroundings, try typing LOOK."




Section - Asking Random Questions

A smarter parser rule when sp_normal (this is the asking random questions rule):
	if input contains "(who|what|where|why|how|do)s?":
		print explanation for asking random questions rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
asking random questions rule	"You can't ask direct questions to the game. Type a verb - noun command like EXAMINE [get noun example] to interact with the story world."


Section - Stripping Verbose Intro

[Things like "I want to go north" or "can you take the ball" ]

A smarter parser rule when sp_normal (this is the stripping verbose intro rule):
	if stripping "i (_be )?(want|will|going)( to)?" is fruitful or stripping "(can|would|will|should|may|try) (i|you|we|to)" is fruitful:
		print explanation for stripping verbose intro rule;
		rule succeeds.

Table of Smarter Parser Messages (continued)
therule			message
stripping verbose intro rule	"You can just begin your command with the verb."


Section - Making assertions


A smarter parser rule when sp_normal (this is the making assertions rule):
	if input contains "(i|he|she|it|this|you|your) (_be|dont|doesnt|cant)":
		print explanation for making assertions rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
making assertions rule	"Please rephrase your command to start with an imperative verb, like LOOK."





Section - Starting command with I

[Normally something like "i don't know what to do" results in the unfortunate message "I only understood you as far as wanting to take inventory."]

A smarter parser rule when sp_normal (this is the starting command with i rule):
	if input starts with "i":
		print explanation for starting command with i rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
starting command with i rule		"Please rephrase your command to start with an imperative verb, like LOOK."




Section - Stripping Adverbs

[There are a few English words that end in "ly" and aren't adverbs, like family and apply. I leave it to a future "Brilliant Parser" extension to deal with these.]

A smarter parser rule when sp_normal (this is the stripping adverbs rule):
	if stripping "\w*ly" is fruitful and the number of words in the rejected command > 1:
		print explanation for stripping adverbs rule;
		rule succeeds.

Table of Smarter Parser Messages (continued)
therule			message
stripping adverbs rule	"A lot of words like prepositions and adverbs are not necessary in interactive fiction."



Section - Unnecessary Movement

["get closer to ball," "move away from ball, "stand next to ball," etc. ]

A smarter parser rule when sp_normal (this is the unnecessary movement rule):
	if input contains "(walk|move|go|stand|get|run) (to|toward|close|away|by)<rs>?" or input contains "(near|nearer|far|farther)":
		print explanation for unnecessary movement rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
unnecessary movement rule		"Usually, you won't need to move within a single location; everything you can see is usually touchable. Use compass directions like [get direction example] to move from one location to another."


Section - Stripping Vague Words


A smarter parser rule when sp_normal (this is the stripping vague words rule):
	if input contains "(some|any|every|no|here|there)(one|thing|body|where|place)?":
		print explanation for stripping vague words rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
stripping vague words rule	"You will need to be more specific. Try typing LOOK to get a description of your surroundings."


Section - Stripping Pointless Words


A smarter parser rule when sp_normal (this is the stripping pointless words rule):
	if stripping "(anyway|instead|very|almost|this|so|just)" is fruitful or stripping "(now|next|around|more)" is fruitful:
		print explanation for stripping pointless words rule;
		if the number of words in the rejected command > 0, rule succeeds;
		else rule fails. [If there are no more words to deal with.]

Table of Smarter Parser Messages (continued)
therule			message
stripping pointless words rule	"A lot of words like prepositions and adverbs are not necessary in interactive fiction."




Section - Stripping Failed With

[">take ball with my hand," for instance]

A smarter parser rule when sp_normal (this is the stripping failed with rule):
	if stripping "(with|using) (the|a|some|my)? ?\w*" is fruitful:
		print explanation for stripping failed with rule;
		rule succeeds.

Table of Smarter Parser Messages (continued)
therule			message
stripping failed with rule	"You don't always need to specify what you're doing something with."


Section - Generic surroundings


A smarter parser rule when sp_normal (this is the generic surroundings rule):
	if input contains "(ground|floor|left|right|above|below|wall|ceiling|forward|backward|onward)s?":
		print explanation for generic surroundings rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
generic surroundings rule		"Look for specific nouns or directions; usually, concepts like the floor and ceiling, or relative directions like left and right, are not relevant."



Section - Stripping Body Parts

[can successfully parse things like "take Phil's arm" or "hit Phil with my hand," and at least prints a recognition of the body part word otherwise. ]


A smarter parser rule when sp_normal (this is the stripping body parts rule):
	replace the regular expression "\b(eye|head|skull|hair|nose|mouth|ear|cheek|forehead)s?\b" in rejected command with "_body";
	replace the regular expression "\b(face|neck|chest|breast|body|nipple|shoulder|collar|arm|wrist)s?\b" in rejected command with "_body";
	replace the regular expression "\b(hand|finger|knuckle|fingernail|waist|thigh|leg|knee)s?\b" in rejected command with "_body";
	replace the regular expression "\b(shin|ankle|foot|feet|toe|elbow|fist|thumb|tongue|lip|heart)s?\b" in rejected command with "_body";
	if stripping "s _body" is fruitful or stripping "(with|my|with my|on|on my) _body" is fruitful:
		print explanation for stripping body parts rule;
		rule succeeds;
	if input contains "_body":
		print explanation for stripping body parts rule;
		rule fails.

Table of Smarter Parser Messages (continued)
therule			message
stripping body parts rule	"You do not normally need to refer to specific parts of the body."




Section - Understood as far as

[Will parse things like "take ball I suppose," "look about the room", or "kill the troll in terror"]

A smarter parser rule when sp_normal (this is the understood as far as rule):
	if the latest parser error is the only understood as far as error:
		if stripping "(the|a|some|my|this|that)" is fruitful, do nothing;
		if input starts with "(l|look|inventory|exit|z|wait)" or input starts with "(yes|no|jump|sorry|sleep)": ["i" is intentionally left out, so this doesn't think something like "I don't think so" is an attempt to take inventory.]
			now rejected command is "[word number 1 in rejected command]";
			print explanation for understood as far as rule;
			rule succeeds;
		if input starts with "(take|get|drop|eat|go|enter|get|x|examine|search|open)" or input starts with "(close|wear|wake|attack|kiss|touch|pull|push|turn)" or input starts with "(squeeze|burn|taste|cut|drink|rub|buy|climb)" and the number of words in rejected command is at least 2:
			now rejected command is "[word number 1 in rejected command] [word number 2 in rejected command]";
			print explanation for understood as far as rule;
			rule succeeds;
		if input starts with "(get off|look under|switch on|switch off|turn on|turn off|take off|listen to|wake up|put on)" and the number of words in rejected command is at least 3:
			now rejected command is "[word number 1 in rejected command] [word number 2 in rejected command] [word number 3 in rejected command]";
			print explanation for understood as far as rule;
			rule succeeds.


Table of Smarter Parser Messages (continued)
therule			message
understood as far as rule		"I only understood the first part of that-- trying anyway."



Chapter - Testing - For Testing Only

Understand "parser" as parser-debugging. Parser-debugging is an action out of world applying to nothing.

SP-debug is a truth state that varies.

Carry out parser-debugging:
	if sp-debug is true:
		now sp-debug is false;
		say "Smart Parser debugging off.";
	otherwise:
		now sp-debug is true;
		say "Smart Parser debugging on."


Smarter Parser ends here.

---- DOCUMENTATION ----


Section: Introduction


Some newcomers to IF are put off by the traditional parser, which offers no default instruction, is fairly rigid about the type of input it will accept, and does not usually phrase error messages in an instructional manner. After one or two confusing error messages, many new players will just give up.

This extension provides a set of responses to various common forms of misunderstood input tried by new players, both understanding a broader range of commands, and instructing players about more appropriate commands when necessary. It also is easy to customize with your own matched patterns.

The easiest way to use the extension is to just include it; nothing else is necessary. Further reading is only necessary if you wish to customize the behavior.

A few notes: This extension is not (as yet) compatible with Cedric Knight's Mistype. It owes a lot conceptually to Emily Short's Inform 6 extension, NewbieGrammar.h, and is also to a large degree a response to its author's experience with new player transcripts (see http://aaronareed.net/wttc/transcripts.html).


Section: Existing Rules


A new rulebook, the "smarter parser" rules, is run just before printing a parser error message. Each rule looks for a specific type of input pattern, and if it finds one, can print a better error message, or even adjust or replace the command and try running it again.

Running the rulebook at the end of the turn cycle rather than the beginning means that nothing in this extension should conflict game-specific code. If you define your own "what is" command, for instance, it will be parsed and dealt with before Smarter Parser has a chance to kick in.

Here is the default rule set, in order, along with an example of the type of malformed command each is designed to catch. Most of these rules can understand a wide range of similar input.

	the standardize apostrophes rule
	the signs of confusion rule
		">I don't know what to do"
	the signs of frustration rule
		">this game sucks"
	the too many words rule
	the stripping niceties rule
		">please take the sword"
	the standardize be verbs rule
	the asking who are you rule
		">who am I talking to "
	the asking who am i rule
		">who am i"
	the asking where am i rule
		">where am i"
	the asking random questions rule
		">why is this sword here"
	the stripping verbose intro rule
		">I want to get the sword"
	the making assertions rule
		">He doesn't like me"
	the starting command with i rule
		">I love this game"
	the stripping adverbs rule	
		">go north carefully"
	the unnecessary movement rule
		">go to sword"
	the stripping vague words rule
		">go somewhere"
	the stripping pointless words rule
		">anyway kill the troll"
	the stripping failed with rule	
		">kill goblin with a rock"
	the generic surroundings rule	
		">look at ground"
	the stripping body parts rule	
		">take woman's hand"
	the understood as far as rule	
		">wait for a while"

( "standardize apostrophes" converts contractions like "I'm" to "I am", then strips any remaining apostrophes; the "standardize be verbs" replaces am, was, were, and so on with "_be". Later rules can use these special tags to simplify pattern matching.)

The rules beginning with "stripping" attempt to remove the offending part of the command and try again: "please take the sword" is tried as "take the sword". (When this happens, the player is informed. You can adjust this behavior by replacing the "announcing new command" rule.)

Rules are tried in the order they are listed. "asking who am i," which would match the line "who am i," needs to come before "asking random questions," which matches the pattern "who (any text)". If the order were reversed, "asking who am i" would never be run. In general, more specific rules involving the same words should come before more general rules.

Each of these rules provides a standard error message when encountered. These can be overwritten by adding to the Table of Extended Smarter Parser Messages, as follows:

	Table of Extended Smarter Parser Messages (continued)
	therule						message
	the stripping niceties rule		"Your fawning attitude sickens me."

To see which rule is generating a particular message, you can type "parser" in an unreleased game.

You can add text before and/or after a Smarter Parser error by changing the values of the indexed texts "pre smart parse text" and "post smart parse text". This might be useful to add brackets or formatting, for instance.

You can remove any of the above rules, or change their order, with standard rule ordering syntax (see the Inform docs chapter on rulebooks for more details):

	The signs of frustration rule is not listed in the smarter parser rulebook.
	The making assertions rule is listed before the stripping niceties rule in the smarter parser rulebook.

You can also remove all of the built-in rules by adding the use option:

	Use empty Smarter Parser rulebook.



Section: Custom Rules


Authors can add new rules to the smarter parser rulebook to extend functionality even farther. These rules use regular expressions to look for patterns in misunderstood player input (see the "Advanced Text" chapter of the Inform docs for more on regular expressions.)

Smarter Parser rules basically take a look at an indexed text called "the rejected command" containing the misunderstood input. The rule can modify or replace this text and succeed, to indicate we should attempt to parse the new command; it can fail, to indicate that we've recognized this input pattern and should not try any further rules; or it can do neither, in which case the next rule in the rulebook will be consulted. 

If no rule succeeds or fails, the standard parser error will be printed (what would have happened anyway if the extension was not included).

If a smarter parser rule succeeds or fails, it needs to print a message informing the player why, since no standard parser error will be printed.

The default Smarter Parser rules make use of a few handy routines to simplify definitions:

	A smarter parser rule (this is the stripping niceties rule):
		if stripping "please" is fruitful:
			print explanation for stripping niceties rule;
			rule succeeds.

The "print explanation for (a rule)" will look up the rule's entry in the Table of Extended Smarter Parser Messages and print it to the screen.

The "if stripping (some text) is fruitful" line will try to cut the given text from the rejected command, and will succeed if it manages to do so.

	A smarter parser rule (this is the asking who am i rule):
		if input contains "who _be i":
			print explanation for asking who am i rule;
			now the rejected command is "examine me";
			rule succeeds.

The "if input contains (some text)" will merely test whether the text is there or not, without changing the rejected command. Note that we then proceed to do so by hand, and since the rule succeeds, the game will immediately try reparsing as if the player had typed "examine me".

	A smarter parser rule (this is the starting command with i rule):
		if input starts with "i":
			print explanation for starting command with i rule;
			rule fails.

You can also use "if input starts with (some text)" and "if input ends with (some text)". Note that the rule fails in this case, indicating we don't want to try re-parsing the command or check any additional rules.

Normally, searches in Smarter Parser rules are restricted to word boundaries, so searching for "go" in "take gold nugget" would fail. You can overrule this behavior by adding ", even within words" to the end of any of these three routines.

A final note: checking regular expressions causes a certain performance hit, so the more rules you add, the slower things will be. On the bright side, the slowdown only happens when the player types misunderstood input.


Example: * Caverns and Kobolds - A tiny scenario to test misunderstood input with.

	*: "Caverns and Kobolds"

	Include Smarter Parser by Aaron Reed.

	Spooky Cave is a room. A glowing sword is in Spooky Cave. Dark Tunnel is north of Spooky Cave. A dusty skull is in Dark Tunnel.

	A smarter parser rule (this is the no leet speak allowed rule):
		if input contains "\w\d+\w":
			print explanation for no leet speak allowed rule;
			rule fails.

	A smarter parser rule (this is the stripping formal address rule):
		if stripping "(sir|maam)" is fruitful:
			print explanation for stripping formal address rule;
			rule succeeds.

	The stripping formal address rule is listed first in the smarter parser rules.

	Table of Extended Smarter Parser Messages (continued)
	therule			message
	no leet speak allowed rule		"This game requires you to use proper spelling and grammar."
	stripping formal address rule		"I appreciate your respect, but all you need to do is type your commands directly."
	
	Test me with "ub4r l33t / what do I do then?? / who am i? / who are you / where am I!? / go to sword / please take the sword sir / this is stupid / why is the sky blue / I want to drop the sword / go somewhere / carefully go north / anyway look around / touch ceiling / touch skull with my foot / scratch head / wait for a while / I like this game after all".
